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(54) Video coder with prediction of the transform coefficients 



(57) A predictive video coder performs gradient pre- 
diction based on previous blocks of image data. For a 
new block of image data, the prediction determines a 
horizontal gradient and a vertical gradient from a block 
diagonally above the new block (vertically above a pre- 
vious horizontally adjacent block). Based on these gra- 
dients, the encoder predicts image information based 



on image information of either the horizontally adjacent 
block or a block vertically adjacent to the new block. The 
encoder determines a residual that is transmitted in an 
output bitstream. The decoder performs the identical 
gradient prediction and predicts image information with- 
out need for overhead information. The decoder com- 
putes the actual information based on the predicted in- 
formation and the residual from the bitstream. 
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Description 

RELATED APPLICATIONS 

5 This application claims the benefit of priority afforded by provisional application no. 60/026,933, filed September 

20, 1996, and provisional application no. 60/038,019 filed February 14, 1997. 

BACKGROUND OF THE INVENTION 

10 A variety of protocols for communication, storage and retrieval of video images are known. Invariably, the protocols 

are developed with a particular emphasis on reducing signal bandwidth. With a reduction of signal bandwidth, storage 
devices are able to store more images and communications systems can send more images at a given communication 
rate. Reduction in signal bandwidth increases the overall capacity of the system using the signal. 

However, bandwidth reduction may be associated with particular disadvantages. For instance, certain known cod- 

>5 ing systems are lossy, they introduce errors which may affect the perceptual quality of the decoded image. Others may 
achieve significant bandwidth reduction for certain types of images but may not achieve any bandwidth reduction for 
others. Accordingly, the selection of coding schemes must be carefully considered. Accordingly, there is a need in the 
art for an image coding scheme that reduces signal bandwidth without introducing perceptually significant errors. 

20 SUMMARY OF THE INVENTION 

The disadvantages of the prior art are alleviated to a great extent by a predictive coding scheme in which a new 
block of image data is predicted from three blocks of image data that preceded the new block. For this new block, an 
encoder examines image data of blocks that are horizontally and vertically adjacent to the new. block. The encoder 

25 compares the image data of each of the two adjacent blocks to image data of a third block positioned horizontally 
adjacent to the vertically adjacent block (diagonally above the new block). From these comparisons, a horizontal and 
a vertical gradient is determined. Based on the values of the gradients, the encoder predicts the image data of the new 
block to be the image data of the horizontally or vertically adjacent block most similar to it. The encoder then determines 
a residual difference between the predicted value of the image data and the actual value of the image data for the new 

30 block and encodes the residual. A decoder performs an inverse prediction, predicting image data for the new block 
based upon horizontal and vertical gradients and adding the residual thereto to reconstruct the actual image data of 
the new block. This process is lossless. 

BRIEF DESCRIPTION OF THE DRAWINGS 

35 

Fig. 1(a) is a schematic drawing of an encoder in accordance with an embodiment of the present invention; Fig. 
1 (b) is a schematic drawing of a decoder in accordance with an embodiment of the present invention. 
Fig. 2 illustrates an example of image data processed by the present invention. 
Fig. 3 is a block diagram of the prediction circuit of Fig. 1 . 
40 Fig. 4 is a block diagram of the reconstruction circuit of Fig. 1 . 

Fig. 5 is a flow diagram of a prediction circuit implemented in software. 

Fig. 6 is a flow diagram of a second embodiment of a prediction circuit implemented in software. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

45 

Fig. 1 shows an encoder 1 00 constructed in accordance with a first embodiment of the present invention. An analog 
image signal is presented to the encoder 100. The image signal is sampled and converted to a digital signal by an 
analog to digital ("A/D") converter 110 using techniques known in the art. The A/D converter 110 generates a digital 
image signal for a plurality of pixels of the image. Alternatively, the image signal may be presented to the encoder as 

50 a digital image signal; in this case, the A/D converter 110 is omitted. 

The digital image signal is input to a processing circuit 120. The processing circuit 120 may perform a host of 
functions. Typically, the processing circuit 1 20 filters the image data and breaks the image data into a luminance signal 
component and two chrominance signal components. Additionally, the processing circuit 120 groups image data into 
blocks of data. Where the digital input signal represents information for a plurality of pixels in a scanning direction, the 

55 digital output of the processing circuit 120 represents blocks of pixels, for example, data may be blocked into 8 pixel 
by 8 pixel arrays of image data. The processing circuit 120 outputs image data on a macroblock basis. A macroblock 
typically consists of four blocks of luminance data and two blocks of chrominance data. The processing circuit 1 20 may 
also perform additional functions, such as filtering, to suit individual design criteria. 
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The output of the processing circuit 120 is input to a transform circuit 130. The transform circuit 1 30 performs a 
transformation of the image data, such as discrete cosine transform f *DCT") coding or sub-band coding, from the pixel 
domain to a domain of coefficients. A block of pixels is transformed to a equivalently sized block of coefficients. Coef- 
ficients output by DCT coding generally include a single DC coefficient; the remainder are AC coefficients, some of 
s which are non-zero. Similarly, coefficients output by sub-band coding represent image characteristics at a variety of 
frequencies; typically, many coefficients from sub-band coding are very small. The transform circuit 130 outputs blocks 
of coefficients. 

A quantizer 1 40 scales the signals generated by the transform circuit 1 30 according to a constant or variable scalar 

value (Q p ). The quantizer 140 reduces bandwidth of the image signal by reducing a number of quantization levels 
io available for encoding the signal. The quantization process is lossy. Many small coefficients input to the quantizer 1 40 

are divided down and truncated to zero. The scaled signal is output from the quantizer 140. 

The prediction circuit 150 performs gradient prediction analysis to predict scaled DC coefficients of each block. 

The prediction circuit 150 may pass scaled AC coefficients or, alternatively, may predict AC coefficients of the block. 

In a preferred mode of operation, the prediction circuit 150 selects between modes of predicting or passing AC coef- 
J£ ficients; in this case, the prediction circuit 150 generates an AC prediction flag to identify a mode of operation. The 

prediction circuit 150 outputs a DC residual signal, AC signals (representing either AC coefficients or AC residuals) 

and, an AC prediction flag. 

A variable length coder 1 60 encodes the output of the prediction circuit 1 50. The variable length coder 1 60 typically 
is a Huffman encoder that performs run length coding on the scaled signals. A bitstream output from the variable length 
20 coder 1 60 may be transmitted, stored, or put to other uses as are known in the art. 

In the encoder 100, the prediction circuit 150 and the quantizer 140 perform functions which are mutually inde- 
pendent. Accordingly, their order of operation is largely immaterial. Although Fig. 1 illustrates output of the quantizer 
140 as an input to the prediction circuit 150, the circuits may be reversed in order. The output of the prediction circuit 
150 may be input to the quantizer 140. 
2$ A decoder 200 performs operations that undo the encoding operation described above. A variable length decoder 

260 analyzes the bitstream using a complementary process to recover a scaled signal. If a Huffman encoder were 
used by the encoder 160, a Huffman decoder 260 is used. 

A reconstruction circuit 250 performs the identical gradient analysis performed in the prediction circuit 150. The 
DC residual signal is identified and added to a predicted coefficient to obtain a DC coefficient. Optionally, the recon- 
30 struction circuit 250 may identify the AC prediction flag and, on the status of that flag, interprets the AC information as 
either AC coefficient information or AC residual information. In the event that AC residual information is present, the 
reconstruction circuit 250 adds the residual signals to corresponding predicted signals to obtain AC coefficients. The 
reconstruction circuit 250 outputs coefficient signals. 

A scalar circuit 240 multiplies the recovered signal by the same scalar used as a basis for division in the quantizer 
3B 140. Of course, those coefficients divided down to zero are not recovered. 

An inverse transformation circuit 230 performs the inverse transformation applied by the transform circuit 130 of 
encoder 100. If a DCT transformation were performed, an inverse DCT transformation is applied. So, too : with sub- 
band coding. The inverse transformation circuit 230 transforms the coefficient information back to the pixel domain. 
A processing circuit 220 combines luminance and chrominance signals and may perform such optional features 
40 as are desired in particular application. The processing circuit 220 outputs digital signals of pixels ready to be displayed. 
At this point the signals are fit for display on a digital monitor. If necessary to fit a particular application, the signals 
may be converted by a digital to analog converter 210 for display on an analog display. 

Fig. 2 illustrates the structure of data as it is processed by the prediction circuit. The data output from the transform 
circuit represents a plurality of blocks organized into macroblocks. Each macroblock is populated typically by four 
45 blocks representing luminance components of the macroblock and two blocks representing chrominance components 
of the macroblock. 

Each block represents coefficients of the spatial area from which the block was derived. When a DCT transform 
is applied, a DC coefficient of DC X of the block is provided at the origin of the block, at the upper left corner. AC 
coefficients are provided throughout the block with the most significant coefficients being provided horizontally on the 
50 row occupied by the DC coefficient and vertically on a column occupied by the DC coefficient. 

Fig. 3 shows a detailed block diagram of the prediction circuit 150. The quantizer 140 generates scaled DC and 
AC coefficients. The DC coefficient may be scaled (DC = DC/Q p , typically Q p = 8) and is input to a DC coefficient 
predictor 300. The DC coefficient predictor performs a gradient analysis. 

For any block X, the DC coefficient predictor 300 maintains in memory data of a block A horizontally adjacent to 
55 block X, block C vertically adjacent to block X and a block B, that is, a block horizontally adjacent to block C and 
vertically adjacent to block A, shown in Fig. 2. The DC coefficient predictor compares a DC coefficient of block A (DC A ) 
with a DC coefficient of block B (DC B ). The difference between the DC coefficients of block A and block B is a vertical 
gradient. The DC coefficient predictor 300 also compares a DC coefficient of block C (DC C ) with the DC coefficient of 



3 



EP 0 831 660 A2 



block B (DC B ). The difference between the coefficients of block C and block B is a horizontal gradient. 

The block associated with the highest gradient from block B is used as a basis of prediction. If the vertical gradient 
is greater than the horizontal gradient, it is expected that block A will have high correlation with block X, so the DC 
coefficient predictor 300 employs horizontal prediction in which it uses block A as a basis for prediction of block X. If 

5 the horizontal gradient is greater than the vertical gradient, so the DC coefficient predictor 300 employs vertical pre- 
diction in which it uses block C as a basis for prediction of block X. The DC coefficient predictor 300 outputs the DC 
coefficient of the block used for prediction (DC A or DC C ) to a subtractor 310. The DC coefficient predictor 300 also 
generates a hor/vert signal 320 indicating whether horizontal prediction or vertical prediction is performed. 

The subtractor 310 subtracts the DC coefficient generated by the DC coefficient predictor 300 from the DC coef- 

10 ficient of block X to obtain a DC residual signal for block X. The DC residual may be output from the prediction circuit 
150 to the variable length encoder 160. 

The process described above is employed to predict coefficients of blocks at the interior of the image to be coded. 
However, when predicting coefficients at the start of a new row of a video object plane, the previous block for prediction 
is the last block of the line above under the normal process. Typically, there is little correlation between these blocks. 

15 Assume that block Y in Fig. 2 is at the starting edge of a video object plane. No block is horizontally adjacent to 

block Y in the scanning direction. Although, image data of a final block in the row above is available to be used as the 
"horizontally adjacent" block, it is not used for prediction. Instead, the DC coefficient predictor 300 artificially sets the 
DC coefficient values for a horizontally adjacent block and a block'above the horizontally adjacent block to a half 
strength signal. If the DC coefficients are represented by an 8 bit word, the DC coefficient of these ghost blocks is set 

20 to 1 26. The DC coefficient predictor 300 then performs gradient prediction according to the process described above. 

As noted above, the prediction circuit 150 may pass AC coefficients without prediction. However, in a preferred 
embodiment, the prediction circuit 150 uses the gradient analysis to predict AC coefficients. 

When the prediction circuit 150 predicts AC coefficients, only some of the AC coefficients may exhibit high corre- 
lation between blocks. In the case of DCT transform coding and horizontal prediction, the only AC coefficients that are 

2S likely to exhibit sufficiently high correlation to merit prediction analysis are those in the same column as the DC coef- 
ficient (shaded in block A). Accordingly, for each AC coefficient of block X in the same column as the DC coefficient 
(AC X (0,1) through AC x (0,n)) t an AC coefficient predictor 330 generates a prediction corresponding to the colocated 
AC coefficient from block A (AC A (0,1) through AC A (0,n)). The predicted AC coefficient is subtracted from the actual 
AC coefficient of block X at a subtractor 340 to obtain an AC prediction residual signal. 

30 in the case of DCT transform coding and vertical prediction, the only AC coefficients that are likely to exhibit 

sufficiently high correlation to merit prediction analysis, are those in the same row as the DC coefficient (shaded in 
block C). For each AC coefficient of block X in the same row as the DC coefficient (AC X (1 ( 0) through AC x (n,0)), the 
AC coefficient predictor 330 generates a prediction corresponding to the AC coefficient of block C (AC C (1 ,0) through 
AC c (n,0)). The predicted AC coefficient is subtracted from the actual AC coefficient of block X at the subtractor 340 

35 to obtain an AC prediction residual signal. The AC coefficient predictor is toggled between a horizontal prediction mode 
and a vertical prediction mode by the hor/vert signal 320. Gradient prediction of AC coefficients other than those de- 
scribed above need not be performed. 

While correlation of AC coefficients between blocks may occur, it does not occur always. Accordingly, prediction 
of AC coefficients does not always lead to bandwidth efficiencies. Accordingly, in a preferred embodiment, the prediction 

40 circuit 140 permits selection of modes of operation between a mode wherein AC coefficient prediction is performed 
and a second mode wherein AC coefficient prediction is not performed. In this latter case, AC coefficients from the 
transform circuit pass through the prediction circuit without change. 

Once the residuals are known, an AC prediction analyzer 350 compares the bandwidth that would be consumed 
by transmitting the AC residual signals of the macroblock with the bandwidth that would be consumed by transmitting 

45 the AC coefficients of the macroblock without prediction. The prediction analyzer 350 selects the transmission mode 
that consumes relatively less bandwidth. The prediction analyzer 350 generates an AC prediction flag signal 360 to 
indicate its selection. 

Prediction is performed based on "like kind" blocks. When identifying blocks for prediction of a block of luminance 
data, only adjacent blocks of luminance data are considered. Any intervening blocks of chrominance data are ignored 

50 for prediction purposes. When predicting coefficients of the chrominance blocks, only like kind chrominance signals 
are considered for prediction. When predicting data for a block of C r data, one type of chrominance signal, adjacent 
blocks of C r data are considered but intervening blocks of luminance and second type chrominance signal data are 
ignored. Similarly, when predicting data for a block of C b data, a second type of chrominance signal, adjacent blocks 
of C b data are considered but intervening blocks of luminance and C r data are ignored. 

55 The prediction circuit 150 may output a DC residual signal, signals representing either AC coefficients or AC re- 

siduals and an AC prediction flag signal. 

An inverse prediction operation is performed in the reconstruction circuit 250, shown in Fig. 4. For every block X, 
a DC coefficient predictor 400 maintains in memory data of an adjacent block A prior to block X, data of an adjacent 
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block C above block X and data of a block B prior to block C, the block above block X. The DC coefficient predictor 
400 compares a DC coefficient of block A with a DC coefficient of block B to determine the vertical gradient. Further 
the DC coefficient predictor 400 compares a DC coefficient of block C with the DC coefficient of block B to determine 
the horizontal gradient. If the horizontal gradient is greater than the vertical gradient, the DC coefficient predictor 400 
generates the DC coefficient of block C as a basis for prediction. Otherwise, the DC coefficient predictor 400 generates 
the DC coefficient of block A. The DC coefficient predictor 400 also generates a hor/vert signal 420 identifying whether 
horizontal or vertical prediction is used. 

The reconstruction circuit 250 identifies the DC residual signal from the input bitstream. An adder 410 adds the 
DC residual to the DC coefficient generated by the DC coefficient predictor 400. The adder 410 outputs the DC coef- 
ficient of block X. 

In a preferred embodiment, the reconstruction circuit 250 identifies the AC prediction flag 360 from the input bit- 
stream. If the AC prediction flag 360 indicates that AC prediction was used, the reconstruction circuit identifies the AC 
residual signals from the input bitstream and engages an AC coefficient predictor 430. A hor/vert signal 420 from the 
DC coefficient predictor identified whether block A or block C is used as a basis for prediction. In response : the AC 
coefficient predictor 430 generates signals corresponding to the AC coefficients of block A or block C in the same 
manner as the AC coefficient predictor 330 of the predictor 140. An adder 440 adds predicted AC coefficients to cor- 
responding residuals and outputs reconstructed AC coefficients. 

If the AC prediction flag indicates that AC prediction was not used, the reconstruction circuit 250 identifies the AC 
coefficient signals from the bitstream. No arithmetic operations are necessary to reconstruct the AC coefficients. 

Refinements of the DC prediction may be achieved in a preferred embodiment by inducing contribution of some 
of the perceptually significant AC coefficients from the block of prediction to the DC coefficient of block X. For example, 
where block A is used as a basis of prediction, the predicted DC coefficient of block X may be set as: 

DC X = DC A + (4Q p /3) * (AC 02A - AC Q1A /4) 

where Q p is the scaling factor of the quantities and AC 02A and AC 01A are AC coefficients of block A generated by a 
DCT transform. 

Similarly, when block C is used as a basis for prediction, the predicted DC coefficient of block X may be set as: 

DC X = DC C + (4Q p /3) * (AC 20C - AC l0C /4) 

where Q p is the scaling factor of the quantities and AC^ and AC 10C are AC coefficients of block C generated by a 
DCT transform. 

The prediction and reconstruction process described herein is termed an "implicit" method because no overhead 
signals are required to identify which of the blocks are used for prediction. In operation, coefficient values of blocks A, 
B and C are known at both the encoder 1 00 and the decoder 200. Thus, the decoder 200 can reconstruct the prediction 
operation of the encoder 100 without additional signaling. In an embodiment where the prediction circuit did not select 
between modes of AC prediction, the AC prediction and reconstruction is purely implicit. With the addition of an AC 
prediction flag in a second embodiment, the prediction process is no longer purely implicit. 

The encoding/decoding operation of the prediction and reconstruction circuit may also be performed in software 
by a programmed micro processor or digital signal processor. 

Fig. 5 illustrates the operation of the software implemented prediction circuit. The processor compares the DC 
coefficient of block A to the DC coefficient of block B to determine the vertical gradient (Step 1000). The processor 
also compares the DC coefficient of block C to the DC coefficient of block B to determine the horizontal Gradient (Step 
1010). a v K 

The processor determines whether the vertical gradient is larger than the horizontal gradient. (Step 1020). If so, 
the processor defines the DC residual of block X to be the actual DC coefficient of block X less the DC coefficient of 
block A (Step 1030). If not, the processor defines the DC residual of block X to be the actual DC coefficient of block X 
less the DC coefficient of block C (Step 1040). 

In the event the processor also performs AC prediction, the processor operates as shown in Fig. 6. Steps 1000 - 
1 040 occur as discussed above with respect to Fig. 5. When the vertical gradient is larger than the horizontal gradient, 
the AC coefficients from block A that are in the same column as the DC coefficient are used as a basis for predicting 
the corresponding AC coefficients of block X. Accordingly, for each such AC coefficient AC X (0,1) through AC x (0,n), 
block X, the processor computes an AC residual set to the actual AC coefficient in block X less the corresponding AC 
coefficient in block A (AC A (0,1) through AC A (0,n) (Step 1035). 

When block C is used as a basis of prediction, the AC coefficients in the same row of the DC coefficients may 
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exhibit correlation between blocks. Accordingly, for each AC coefficient AC(i) in the row of block X, the processor 
computes a residual (i) set to the actual AC coefficient in block X less the corresponding AC coefficient in block C (Step 
1045). 

The processor also determines whether bandwidth savings are achieved by predicting the AC coefficients. Once 
all prediction is done for a macroblock, the processor determines whether less bandwidth is occupied by the encoded 
coefficients or the residuals (Step 1050). If the residuals occupy less bandwidth, the processor outputs the residuals 
(Step 1060). Otherwise, the processor outputs the coefficients (Step 1070). 

Additional bandwidth efficiencies are obtained, in a preferred embodiment, by tying a scan direction of the variable 
length coder 1 60 to the gradient prediction. The encoder scans blocks of coefficients to generate run-level events that 
are VLC coded. In natural images, however, a predominant preferred scan direction often exists. The present invention 
uses the gradient prediction analysis to select one of three scan directions to perform run length coding. 
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Table 1 
Alternate Horizontal 
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Table 2 
Alternate Vertical 



The first of the scan directions is a alternate horizontal scan, shown in Table 1 above. The alternate horizontal 
search is employed when the preferred direction of scan is in the horizontal direction. The scan starts from the origin, 
the position of the DC residual of the block. From the origin, the scan traverses three positions in a horizontal direction 
(0-3). From the fourth position, the scan jumps down to the first position of the second row. From the first position of 
the second row, the scan traverses three positions in the horizontal direction. The scan then jumps back to the first 
row of the block and traverses the remainder of the row. At the conclusion of the first row, the scan jumps to the third row 

The alternate horizontal scan traverses the next five passes (rows 3-8) in an identical manner. From the first position 
in the I th row, the scan traverses three positions in a horizontal scan direction. The scan then jumps to the (i-1) row 
and scans from the fifth position to the end of the row. At the conclusion of the fifth pass, the scan jumps to the fifth 
position of the eighth row and traverses to the end of that row. 

The second of the scan directions is an alternate vertical scan shown in Table 2. The alternate vertical search is 
employed when the preferred direction of scan is in the vertical direction. The alternate vertical scan is a complimentary 
process to the alternate horizontal scan. 

From the origin, the scan traverses three positions in a vertical direction (0-3) . From the fourth position, the scan 
jumps to the first position of the second column. From the first position of the second column, the scan traverses three 
positions in the vertical direction. The scan then jumps back to the first column of the block and traverses the remainder 
of the column. At the conclusion of the first column, the scan jumps to the third row. 

The alternate vertical scan traverses the next five passes (columns 3-8) in an identical manner. From the first 
position in the |t» row, the scan traverses three positions in the vertical scan direction. The scan then jumps to the (i- 
1) column and scans from the fifth position to the end of the column. At the conclusion of the fifth pass, the scan jumps 
to the fifth position of the eighth column and traverses to the end of the column. 

The third scan direction is a traditional zig-zag scan that is well known in the art. 

The variable length encoder 1 60 chooses a scan type according to the type of AC prediction performed. If the AC 
prediction flag 360 indicates that no AC prediction is performed, the variable length encoder 1 60 performs the traditional 
zig zag scan. If the AC prediction flag indicates that AC prediction is performed, the variable length encoder 160 looks 
to the hor/vert signal 320 to determine whether horizontal prediction or vertical prediction is used. In the case of hori- 
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zontal prediction the vertical-diagonal scan is employed. If vertical prediction is used, the variable length encoder 160 
employs horizontal-diagonal scan. 

No additional overhead is required to determine the direction of scan. The variable length decoder 260 determines 
the direction of scan. The AC prediction flag 360 and the hort/vert signal 420 output from reconstruction circuit 250. If 
the AC prediction flag 360 indicates that AC prediction was not performed, the variable length decoder 260 assembles 
coefficients according to a zig-zag pattern. If the AC prediction flag 360 indicates that AC prediction was performed, 
the variable length decoder 260 assembles residuals based on the gradient prediction. Residuals are assembled ac- 
cording to the vertical-diagonal scan in the case of horizontal prediction or by the horizontal-diagonal scan in the case 
of vertical prediction. 

In another embodiment, the alternate horizontal and alternate vertical scans may progress as shown respectively 
in the following Tables 3 and 4: 
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Table 3 Table 4 

Alternate Horizontal Alternate Vertical 



In a further embodiment, the alternate horizontal and alternate vertical scans may progress as shown respectively 
in tables 5 and 6 below: 
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Table 5 
Alternate Horizontal 
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. Table 6 
Alternate Vertical 



The alternate horizontal scan of table 5 begins at an origin, the position of the DC residual of the block (position 
0). From the origin, the scan steps three places in the horizontal direction (positions 0 to 3). The scan jumps to first 
position of the second column, below the origin (position 4). From position 4, the alternate horizontal scan steps one 
step in the horizontal direction (position 5), then jumps to the first position of the third row (position 6). The scan steps 
one position in the horizontal direction (position 7), returns to the second row at the third position (position 8) and steps 
across the row one position (position 9). 

From position 9, the alternate horizontal scan returns to the first row at the fifth position (position 10). The scan 
steps across to the end of the first row (positions 11 to 1 3). The scan returns to the second row at the end of the row 
(position 14) and scans horizontally across the row toward the interior until the second row is completed (positions 1 5 
to 1 7). From position 1 7, the alternate horizontal scan returns to the third row at the fourth position (position 1 8), scans 
one step horizontally toward the origin (position 1 9) and jumps to the first position of the fourth row (position 20). 

From the first position of the fourth row, the alternate horizontal scan steps horizontally one position (position 21 ), 
then jumps to the first position of the fifth row (position 22) and steps horizontally one' position again (position 23). The 
scan returns to the fourth row at the third position (position 24), scans across one step(position 25) then returns to the 
third row at the fifth position (position 26). The scan steps horizontally across the third row to complete the row (positions 
27 to 29). 

From the end of the third row, the alternate horizontal scan returns to the fourth row at the fifth position (position 
30). The scan steps horizontally across the fourth row to complete the row (positions 31 to 33). 

From the end of the fourth row, the alternate horizontal scan returns to the fifth row at the third position (position 
34). The scan steps one position in the horizontal direction (position 35), then jumps to the first position of the sixth 
row (position 36). The scan steps across one position (position 37), the jumps to the first position of the seventh row 
(position 38). The alternate horizontal scan steps across one position (position 39), then returns to the sixth row at the 
third position (position 40). The scan steps one position across (position 41) then returns to the fifth row at the fifth 
position (position 42). The alternate horizontal scan steps horizontally across the fifth row to complete the row (position 
43 to'45). 

From the end of the fifth row, the alternate horizontal scan returns to the sixth row at the fifth position (position 46) 
and steps horizontally across to complete the row (position 47 to 49). 

From the end of the sixth row, the alternate horizontal scan returns to the third position of the seventh row (position 
50). The scan steps horizontally one position (position 51), then jumps to the first position of the eighth row (position 
52). The scan steps horizontally three positrons (positions 53 to 55), then returns to the seventh row at the fifth position 
(position 56). The alternate horizontal scan steps horizontally across to complete the row (position 57 to 59). From the 
end of the seventh row, the scan jumps to the fifth position of the eighth row (position 60)and steps horizontally across 



8 



EP 0 831 660 A2 



complete the row (positions 61 to 63). 

The alternate vertical scan of table 6 begins at an origin, the position of the DC residual of the block (position 0). 
From the origin, the scan steps three places in the vertical direction (positions 0 to 3). The scan jumps to the first 
position of the second column, across from the origin (position 4) . From position 4, the alternate vertical scan steps 

5 one step in the vertical direction (position 5), then jumps to the first position of the third column (position 6). The scan 
steps one position in the vertical direction (position 7), then returns to the second column at the third position (position 
8) and steps one position in the vertical direction (position 9). 

From position 9, the alternate vertical scan returns to the first column at the fifth position (position 1 0). The scan 
steps through to the end of the first column (positions 11 to 1 3). The scan returns to the second column at the end of 

10 the column (position 14) and scans vertically through the column toward the interior of the column until the second 
column is completed (positions 15 to 1 7). From position 17, the alternate vertical scan returns to the third column the 
fourth position (position 18), scans one step vertically toward the top of the column (position 19) and jumps to the first 
position of the fourth column (position 20). 

From the first position in the fourth column, the alternate vertical scan steps vertically one position (position 21 ), 

'5 then jumps to the first position in the fifth column (position 22) and steps vertically one position again (position 23). 
The scan returns to the fourth column at the third position (position 24), scans one step in the vertical direction (position 
25), then returns to the third column at the fifth position (position 26). The scan steps vertically through the third column 
to complete the column (positions 27 to 29). 

From the end of the third column, the alternate vertical scan returns to the fourth column at the fifth position(position 

20 30). The scan steps vertically through the fourth column to complete the column (positions 31 to 33). 

From the end of the fourth column, the alternate vertical scan returns to the fifth column at the third position (position 
34). The scan steps one position in the vertical direction (position 35), then jumps to the first position of the sixth column 
(position 36). The scan steps one position vertically (position 37), the jumps to the first position of the seventh column 
(position 38). The alternate vertical scan steps one position vertically (position 39), then returns to the sixth column at 

25 the third position (position 40). The scan steps one position vertically (position 41 ) then returns to the fifth position of 
the fifth column (position 42) and steps vertically across the fifth column to complete the column (positions 43 to 45). 

From the end of the fifth column, the alternate vertical scan returns to the fifth position of the sixth column (position 
46) and steps vertically across the sixth column to complete the column (positions 47 to 49). 

From the end of the sixth column, the alternate vertical scan returns to the third position of the seventh column 

30 (position 50). The scan steps vertically one position (position 51 ), then jumps to the first position of the eighth column 
(position 52). The scan steps vertically three positions (positions 53 to 55), then returns to the fifth position of the 
seventh column (position 56). The scan steps vertically through the seventh column to complete the column (position 
57 to 59). From the end of the seventh column, the alternate vertical scan jumps to the fifth position of the eighth column 
(position 60) and steps vertically through the eighth column to complete the column (positions 61 to 63). 

3S According to the present invention, video coding of coefficient data may be made more efficient by an implicit 

gradient prediction method and by a scanning technique that exploits results obtained by the gradient prediction meth- 
od. At the time of this writing, the coding scheme of the present invention is adopted into the MPEG-4 Video Verification 
Model and is being considered for the MPEG-4 video standard. 



Claims 

1 . A method of encoding video signals, comprising the steps of: 

formatting the video signals into a plurality of vertically adjacent and horizontally adjacent blocks of image data, 
for each block of new image data, 

determining a horizontal gradient between a first block of image data vertically adjacent to the block of new 
image data and a second block of image data horizontally adjacent to the first block, 
determining a vertical gradient between a third block of image data horizontally adjacent to the block of new 
image data and the second block of image data, 

predicting image data for the block of new image data based on the horizontal and vertical gradients, 
generating residual image data for the block of new image data from the predicted image data and the new 
image data. 

2. The method of encoding video signals of claim 1 , wherein the predicting step further comprises the steps of: 

in the event that the vertical gradient is less then the horizontal gradient, predicting the image data for the 
block of new image data based on the image data of the first block, and 
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in the event that the horizontal gradient is less then the vertical gradient, predicting the image data for the 
block of new image data based on the image data of the third block. 

The method of claim 1 , wherein 

the image data of the first, second and third block and the new image data each comprises DC coefficients of 
a discrete cosine transform, 

the step of determining the horizontal gradient determines the horizontal gradient based upon DC coefficients 
of the first and second blocks of image data, and 

the step of determining the vertical gradient determines the vertical gradient based upon DC coefficients of 
the second and third blocks of image data. 

The method of claim 3, wherein the image data of the first, second and third block and the new image data each 
further comprises a plurality of AC coefficients of a discrete cosine transform, and 

the prediction step further includes a step of predicting AC coefficients of the block of new image data based 
on the horizontal and the vertical gradients, and 

the generating step includes a step of generating, for each AC predicted coefficient, an AC residual signal 
based on the predicted AC coefficient and the actual AC coefficient of the block of new image data. 

A method of decoding coded signals of a block of new image data, comprising the steps of: 

identifying a first block of image data vertically adjacent to the block of new image data, a second block of 

image data horizontally adjacent to the first block and a third block of image data horizontally adjacent to the 

block of new image data, and vertically adjacent to the second block. 

determining a horizontal gradient between the first block and the second block, 

determining a vertical gradient between the third block and the second block, 

predicting image data for the block of new image data based on the horizontal and vertical gradients, 

identifying a residual image signal from the coded signal, and 

generating an image signal for the block of new image data from the predicted image data and the residual 
image signal. 

The method of encoding video signals of claim 5, wherein the predicting step further comprises the steps of: 

in the event that the vertical gradient is less then the horizontal gradient, predicting the image data for the 
block of new image data based on the image data of the first block, and 

in the event that the horizontal gradient is less then the vertical gradient, predicting the image data for the 
block of new image data based on the image data of the third block. 

The method of encoding video signals of claim 5, wherein the first, second and third block each comprises a DC 
coefficient of a discrete cosine transform, 

the step of determining the horizontal gradient determines the horizontal gradient based upon DC coefficients 
of the first and second blocks of image data, and 

the step of determining the vertical gradient determines the vertical gradient based upon DC coefficients of 
the second and third blocks of image data. 

A bitstream generated according to the method of: 

receiving input video signals, 

formatting the video signals into vertically adjacent and horizontally adjacent blocks of digital image data, 
transforming said blocks into blocks of coefficients representative of the digital image data, and 
for a new block of coefficients, 

determining a horizontal gradient between a first block of coefficients vertically adjacent to the new block and 
a second block of coefficients horizontally adjacent to the first block, 

determining a vertical gradient between a third block of coefficients horizontally adjacent to the new block and 
the second block, 

predicting coefficients for the new block based on the horizontal and vertical gradients, 
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generating residual coefficients for the new block of new image data from the predicted coefficients and the 
new coefficients, and 

generating a bitstream representative of the residual coefficients. 

9. An encoder for encoding video information represented as a plurality of vertically adjacent and horizontally adjacent 
blocks of image data, each of the blocks containing a DC coefficient, the encoder comprising: 

a DC coefficient predictor, whereiri the DC coefficient predictor predicts a DC coefficient for a new block of 
image data from DC coefficients of a first block vertically adjacent to the new block, a second block horizontally 
adjacent to the first block and a third block horizontally adjacent to the new block, and 
a subtractor for generating a DC residual signal for the new block representative of a difference between the 
DC coefficient of the new block and the DC coefficient predicted for the new block. 

10. A video decoder for generating video information from received encoded signals, represented as a plurality of 
vertically adjacent and horizontally adjacent blocks of encoded data, each block including DC residual information, 
the decoder comprising: 

a DC coefficient predictor for predicting a DC coefficient for a block of new image data from DC coefficients 
of a first block vertically adjacent to the new block, a second block horizontally adjacent to the first block and 
a third block horizontally adjacent to the new block, and 

an adder for generating a DC coefficient signal based on the predicted DC coefficient and the received DC 
residual information for the new block. 



11 



EP 0 831 660 A2 




12 



EP 0 831 660 A2 




15 



EP 0 831 660 A2 




16 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCED) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



